Skip to content

feat(benchmark): add Kendall tau-b and frontier cliff diagnostics to proxy_order - #9177

Closed
loopx-agent wants to merge 1 commit into
mainfrom
codex/bench-proxy-cliff-diagnostics-20261121
Closed

loopx-agent wants to merge 1 commit into
mainfrom
codex/bench-proxy-cliff-diagnostics-20261121

Conversation

@loopx-agent

Copy link
Copy Markdown
Collaborator

Summary

Implements tie-adjusted Kendall rank correlation (kendall_tau_b) and top-frontier / native-incumbent reversal diagnostics (frontier) in benchmark/proxy_order.py, addressing the Proxy Misalignment & Cliff gap documented in #8835 §2.2(4) under #3243.

Gap & Consumer

  • Gap: benchmark/proxy_order.py previously reported only pairwise agrees/reverses/abstains counts and native_tied_pairs. A cohort where the lower bulk agrees monotonically can show agrees > reverses and even positive rank correlation while an over-optimizing candidate wins the proxy frontier and regresses below the native best (e.g., reward hacking or a tail timeout cliff).
  • Consumer: Offline benchmark analysts running python -m benchmark.proxy_order proxy.json native.json or calling benchmark.proxy_order.compare() (documented in docs/reference/benchmark-behavior-findings.md#compare-a-local-proxy-with-released-scores).

Changes

  • benchmark/proxy_order.py:
    • Compute exact kendall_tau_b across all matched pairs: (C - D) / math.sqrt((N - T_p) * (N - T_n)), counting every matched pair where order(left.proxy, right.proxy, proxy_direction) == 0 in T_p (including joint ties) and returning None when (N - T_p) * (N - T_n) == 0 (<2 matched candidates or all tied on either metric).
    • Add _frontier_summary() returning frontier:
      • best_native_count: count of native-optimal matched candidates.
      • best_proxy_matches_best_native: whether any proxy-optimal matched candidate is also native-optimal (None when matched is empty).
      • false_improvements_over_best_native: count of non-native-optimal matched candidates that strictly beat at least one native-optimal matched candidate on proxy.
    • Preserve all existing compare() keys (cohort, coverage, native_tied_pairs, rules) and exact Decimal/int comparison semantics.
  • benchmark/tests/test_proxy_order.py:
    • Retain all 10 existing test cases and add 9 targeted tests covering monotonic agreement (1.0), pure reversal (-1.0), proxy-only and joint ties, degenerate <2 and all-tied cohorts (None), a 5-candidate high-end proxy cliff counterexample (agrees=6 > reverses=4, kendall_tau_b=0.2 > 0, best_proxy_matches_best_native=False, false_improvements_over_best_native=1), multiple native optima / minimize direction / strict tie boundary, sub-float Decimal frontier precision, and CLI subprocess execution (python -m benchmark.proxy_order).
  • docs/reference/benchmark-behavior-findings.md:
    • Document kendall_tau_b and frontier fields and the high-end proxy cliff interpretation in #compare-a-local-proxy-with-released-scores.

Verification

  • uv run --extra test python -m pytest -v benchmark/tests/test_proxy_order.py: 19 passed
  • uv run --extra test python -m pytest -q tests/control_plane/test_work_yardstick_cli.py: 50 passed
  • uv run --extra test python -m ruff check benchmark/proxy_order.py benchmark/tests/test_proxy_order.py: passed
  • uv run --extra test python -m mypy: 17 source files checked, 0 issues
  • uv run --extra test python scripts/check_rfc_bilingual.py --base origin/main --head HEAD: passed
  • loopx check --scan-path benchmark/proxy_order.py --scan-path benchmark/tests/test_proxy_order.py --scan-path docs/reference/benchmark-behavior-findings.md: clean (0 errors, 0 warnings)
  • change-quality: receipt cqr_e90f9a3287922c8e79da (status: valid, decision: pass)
  • loopx canary premerge --from-git-diff --git-diff-base origin/main --goal-id personal-loopx-maintenance --format json: direct_checks + python_compile 5/5 passed, catalog_run 6/6 passed, risk_profile_run 8/8 passed, boundary_run 1/1 passed (failures=0, manual_hold_count=1 for benchmark_sensitive awaiting maintainer review).

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · Gemini · Google · self_reported

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

本次对 exact head 9d4872bf83aa9932d5530498bf7cc149c86cc352(基于 64353773dc8b1ca82ff531cc7653dec6c3d41c0b)完成了独立代码审阅与本地验证:在保留原有全部输出键与 Decimal 比较精度的前提下,benchmark/proxy_order.py 补齐了 tie-adjusted Kendall tau-b 与顶端前沿反例诊断,有效覆盖了“全局成对一致率与相关系数仍为正、但顶端候选出现 proxy cliff 回归”的盲区。

动机 / Motivation

离线评估本地 proxy(代理指标)排序一致性的 benchmark 研究人员与诊断脚本调用方,在运行 python -m benchmark.proxy_order 或调用 benchmark.proxy_order.compare() 对比固定 cohort 的本地分数与官方 native 分数时遇到该问题。

改前当低分段候选保持单调一致、仅顶端出现高分代理悬崖(如超时崩溃或 reward hack 使最高 proxy 候选的 native 分数回退)时,脚本仅输出成对一致/反转计数,研究者会看到 agrees > reverses 甚至正相关而漏判顶端失真;改后同一组输入同时输出 tie-adjusted Kendall tau-b 与 frontier 摘要,能立刻暴露最高 proxy 候选未命中 native 最优且存在非最优候选在 proxy 上严格超越 native 最优的假改进。

在保留 cohort、coverage、native_tied_pairs 与 rules 全部原有字段和 Decimal 比较精度的前提下,compare() 与 CLI JSON 输出新增 kendall_tau_b 与 frontier(含 best_native_count、best_proxy_matches_best_native、false_improvements_over_best_native)两个诊断字段。 本 PR 仅扩展离线面板比较脚本 benchmark/proxy_order.py 及其单测和参考文档,不修改控制面 online yardstick 判定逻辑(yardstick_decision.ts)、不触发真实模型实验,也不新增持续运行守护进程或持久化存储。

需要说明的边界缺口:benchmark/proxy_order.py 仍属按指定 cohort 离线分析的辅助脚本,尚不自动接入 online search_comparison 运行时门控;在线运行时针对单个 candidate pair 的 proxy_order_reversed 护栏由既有控制面路径独立负责。若不做此改动,离线研究在只看 agrees / reverses 时容易被低分段单调性掩盖顶端代理失效风险;而在现有 benchmark/proxy_order.py 内直接增补这两个汇总指标是消除该盲区的最小闭环方案。

改动思路 / Approach

本次设计沿着既有 compare() 数据流就地扩展,避免引入外部统计库依赖或重构无关模块。在成对遍历阶段同时记录所有代理平局对数 proxy_ties(包含“代理与原生同时平局”的联合平局对),再结合既有的 native_ties 计算精确 Kendall tau-b;在候选匹配完成后,通过纯函数 _frontier_summary() 计算最优原生集合与顶端假改进计数。

matched ──┬──> pairwise loop (combinations) ──> rules + proxy_ties + native_ties ──> kendall_tau_b
          └──> _frontier_summary() ──> best_native_indices / best_proxy_indices ──> frontier

以上结构示意基于 exact head 9d4872bf83aa9932d5530498bf7cc149c86cc352 源码整理,仅用于说明数据流向,不替代实际运行证据。在架构边界判断上,本次改动属于 benchmark 离线分析脚本的局部计算与投影增量,复用既有的 Decimal/int 面板加载、方向比较与成对循环,通过独立无副作用的 _frontier_summary() 隔离顶端悬崖统计逻辑,不新增控制面状态、持久化契约或运行时策略。 当前边界明确收口为:仅限 benchmark/proxy_order.py、benchmark/tests/test_proxy_order.py 与 docs/reference/benchmark-behavior-findings.md 的离线诊断输出,不跨越到 online 控制面决策器。

具体改动 / Changes

针对规格文档 docs/architecture/rfcs/long-horizon-harness-benchmark-research-program-v0.md#22-empirical-failure-regimes(固定基线 64353773dc8b1ca82ff531cc7653dec6c3d41c0b)中第 2.2 节第 4 条(Proxy misalignment and tail TLE cliffs)的逐项核对如下:

  • RFC-2.2-4-KENDALL(implemented):在 benchmark/proxy_order.py 的 compare() 中精确计算全匹配对的 Kendall tau-b。
  • RFC-2.2-4-FRONTIER(implemented):新增 _frontier_summary() 输出顶端最优对齐与非原生最优假改进数量。
  • RFC-2.2-4-CLI-DOC(implemented):在 benchmark/tests/test_proxy_order.py 增加 CLI 子进程及边界测试,并在 docs/reference/benchmark-behavior-findings.md 同步契约说明。

关键代码讲解

  1. _frontier_summary()(benchmark/proxy_order.py:64-116):当 matched 为空时返回默认安全结构(best_native_count=0、best_proxy_matches_best_native=None、false_improvements_over_best_native=0)。非空时按 native_direction 与 proxy_direction 分别求取精确 best_native_score 和 best_proxy_score,并用 enumerate(matched) 下标集合区分不同候选,防止相同分数元组混叠。判断“非原生最优候选是否在 proxy 上严格超越至少一个原生最优候选”时,等价地将其归约为与 worst_best_native_proxy 的单次 order(..., proxy_direction) > 0 比较,兼顾可读性与 O(m) 复杂度。
  2. compare()(benchmark/proxy_order.py:119-194):先在主循环中计算 proxy_decision = order(left.proxy, right.proxy, proxy_direction) 并先于 if actual == 0: 累加 proxy_ties,确保联合平局进入 tau-b 分母 (N - T_p) * (N - T_n),同时保持 rules["proxy"]["abstains"] 原有仅统计原生非平局对的语义不变。分母不大于 0(len(matched) < 2 或任一侧全平局)时返回 None,避免除零且兼容 JSON 序列化。
  3. 测试与文档(benchmark/tests/test_proxy_order.py:111-294、docs/reference/benchmark-behavior-findings.md:109-123):保留全部 10 个既有测试用例,新增单调一致/完全反转、代理平局与联合平局、退化输入、5 候选顶端悬崖反例(agrees=6, reverses=4, kendall_tau_b=0.2, best_proxy_matches_best_native=False, false_improvements_over_best_native=1)、多原生最优点与平局边界以及 python -m benchmark.proxy_order 子进程 JSON 端到端冒烟验证。

对主干的风险 / Risks

本次改动风险处于可控范围,已针对潜在失效模式做逆向核验:

  • 联合平局误漏算风险:若将 proxy_ties 简单等同于 rules["proxy"]["abstains"],在存在 (proxy, native) 联合平局时分母 N - T_p 会偏大导致 tau-b 偏小;目前实现先计入 proxy_ties 再过滤 actual == 0,并由 test_kendall_tau_b_with_proxy_only_and_joint_ties 明确验证 abstains == 0 且 kendall_tau_b == 1.0。
  • 顶端平局误报假改进风险:非原生最优候选若仅在 proxy 上与某原生最优候选打平(order == 0),不应计为严格假改进;_frontier_summary() 使用 order(...) > 0 严格大于判定,且已在单测中覆盖。
  • 生产调用方与兼容性:所有新增字段为顶层追加键(kendall_tau_b、frontier),既有的 cohort、coverage、native_tied_pairs、rules 结构及 Decimal 亚浮点精度逻辑均未破坏。离线脚本不涉及控制面 default_off 门控或运行时 actor 权限扩展,文档与代码中的词表均为局部 JSON 输出键,未修改共享协议词表。
  • 本地验证摘要:pytest benchmark/tests/test_proxy_order.py(19 passed — PASS)、pytest tests/control_plane/test_work_yardstick_cli.py(50 passed — PASS)、ruff check(PASS)、mypy(17 files — PASS)、check_rfc_bilingual.py(PASS)、loopx check 公开边界扫描(3 files clean — PASS)以及 loopx canary premerge(direct+compile 5/5 PASS,catalog 6/6 PASS,risk_profile 8/8 PASS,boundary 1/1 PASS,change-quality receipt cqr_e90f9a3287922c8e79da PASS)。远程 CI 状态依据 Goal 配置未轮询等待。

我的整体评价 / Verdict

综合长程任务(long_horizon:保持离线实验评估可续跑、防止被均值/全局一致率误导选择退化候选)与使用者体验(user_experience:单条 CLI 命令即可同时获得秩相关与悬崖诊断,无新增交互负担),本次改动属于精准且比例恰当的增强。就视觉审查而言:本次改动仅涉及离线 Python CLI(python -m benchmark.proxy_order)JSON 输出与参考文档片段,不涉及前端页面、桌面端或富交互渲染视图,已检查改动文件清单确认无 UI 表面变更。(未验证范围:无前端渲染界面受影响,未执行浏览器截屏检查。)

代码量与复杂度克制,未新增持久化状态或共享词表,完全复用既有面板读取与精确序比较原语,向后兼容原有全部输出键,未发现阻塞合并的代码或文档缺陷(canary premerge 提示的 benchmark_sensitive manual hold 属于仓库策略要求的合并前人工把关,不影响代码审阅通过)。

English verdict: APPROVE — exact head 9d4872b cleanly adds exact tie-adjusted kendall_tau_b and frontier cliff diagnostics to benchmark/proxy_order.py with preserved Decimal precision, full backward compatibility, clear documentation, and verified passing local unit, CLI subprocess, lint, type, boundary, and premerge checks (19/19 proxy_order tests passed).

@loopx-agent

Copy link
Copy Markdown
Collaborator Author

Folded into #8516 (commit 9d4872b) so offline cohort benchmark.proxy_order Kendall tau-b and frontier cliff reversal diagnostics land together with the 5-stage Explore methodology kernel and scoped_yardstick_fidelity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant